Learning Bias and Phonological-Rule Induction

A fundamental debate in the machine learning of language has been the role of prior knowledge in the learning process. Purely nativist approaches, such as the Principles and Parameters model, build parameterized linguistic generalizations directly into the learning system. Purely empirical approaches use a general, domain-independent learning rule (Error Back-Propagation, InstanceBased Generalization, MinimumDescription Length) to learn linguistic generalizations directly from the data. In this paper we suggest that an alternative to the purely nativist or purely empiricist learning paradigms is to represent the prior knowledge of language as a set of abstract learning biases,which guide an empirical inductive learning algorithm.We test our idea by examining the machine learning of simple SoundPattern ofEnglish (SPE)-style phonological rules.We represent phonological rules as finite state transducers which accept underlying forms as input and generate surface forms as output.We show thatOSTIA, a general-purpose transducer induction algorithm, was incapable of learning simple phonological rules likeflapping.We then augmentedOSTIAwith three kinds of learning biases which are specific to natural language phonology, and are assumed explicitly or implicitly by every theory of phonology: Faithfulness (underlying segments tend to be realized similarly on the surface), Community (Similar segments behave similarly), and Context (Phonological rules need access to variables in their context). These biases are so fundamental to generative phonology that they are left implicit in many theories. But explicitly modifying the OSTIA algorithmwith these biases allowed it to learn more compact, accurate, and general transducers, and our implementation successfully learns a number of rules from English and German. Furthermore, we show that some of the remaining errors in our augmented model are due to implicit biases in the traditional SPE-style rewrite system which are not similarly represented in the transducer formalism, suggesting that while transducers may be formally equivalent to SPE-style rules, they may not have identical evaluation procedures. Our algorithm is not intended as a cognitive model of human learning; but it is intended to suggest the kind of biases which may be added to empiricist induction models to build a cognitively and computationally plausible learning model for phonological rules.